Registration of Multisensor Images through a Conditional Generative Adversarial Network and a Correlation-Type Similarity Measure

The automatic registration of multisensor remote sensing images is a highly challenging task due to the inherently different physical, statistical, and textural characteristics of the input data. Information-theoretic measures are often used to favor comparing local intensity distributions in the images. In this paper, a novel method based on the combination of a deep learning architecture and a correlation-type area-based functional is proposed for the registration of a multisensor pair of images, including an optical image and a synthetic aperture radar (SAR) image. The method makes use of a conditional generative adversarial network (cGAN) in order to address image-to-image translation across the optical and SAR data sources. Then, once the optical and SAR data are brought to a common domain, an area-based ℓ2 similarity measure is used together with the COBYLA constrained maximization algorithm for registration purposes. While correlation-type functionals are usually ineffective in the application to multisensor registration, exploiting the image-to-image translation capabilities of cGAN architectures allows moving the complexity of the comparison to the domain adaptation step, thus enabling the use of a simple ℓ2 similarity measure, favoring high computational efficiency, and opening the possibility to process a large amount of data at runtime. Experiments with multispectral and panchromatic optical data combined with SAR images suggest the effectiveness of this strategy and the capability of the proposed method to achieve more accurate registration as compared to state-of-the-art approaches

Introducing temporal correlation in rainfall and wind prediction from underwater noise

While in the past the prediction of wind and rainfall from underwater noise was performed using empirical equations fed with very few spectral bins and fitted to the data, it has recently been shown that regression performed using supervised machine learning techniques can benefit from the simultaneous use of all spectral bins, at the cost of increased complexity. However, both empirical equations and machine learning regressors perform the prediction using only the acoustic information collected at the time when one wants to know the wind speed or the rainfall intensity. At most, averages are made between spectra measured at subsequent times (spectral compounding) or between predictions obtained at subsequent times (prediction compounding). In this article, it is proposed to exploit the temporal correlation inherent in the phenomena being predicted, as has already been done in methods that forecast wind and rainfall from their values (and sometimes those of other meteorological quantities) in the recent past. A special architecture of recurrent neural networks, the long shortterm memory, is used along with a data set composed of about 16 months of underwater noise measurements (acquired every 10 min, simultaneously with wind and rain measurements above the sea surface) to demonstrate that the introduction of temporal correlation brings significant advantages, improving the accuracy and reducing the problems met in the widely adopted memoryless prediction performed by random forest regression. Working with samples acquired at 10-min intervals, the best performance is obtained by including three noise spectra for wind prediction and six spectra for rainfall prediction

Evaluating LoRaWAN connectivity in a marine scenario

The growing need for interoperability among the different oceanic monitoring systems to deliver services able to answer the requirements of stakeholders and end-users led to the development of a low-cost machine-to-machine communication system able to guarantee data reliability over marine paths. In this framework, an experimental evaluation of the performance of long-range (LoRa) technology in a fully operational marine scenario has been proposed. In-situ tests were carried out exploiting the availability of (i) a passenger vessel and (ii) a research vessel operating in the Ligurian basin (North-Western Mediterranean Sea) both hosting end-nodes, and (iii) gateways positioned on mountains and hills in the inland areas. Packet loss ratio, packet reception rate, received signal strength indicator, signal to noise, and expected signal power ratio were chosen as metrics in line of sight and not the line of sight conditions. The reliability of Long Range Wide Area Network (LoRaWAN) transmission over the sea has been demonstrated up to more than 110 km in a free space scenario and for more than 20 km in a coastal urban environment

A Tiling-Based Strategy for Large-Scale Multisensor Optical-Sar Image Registration

The automatic registration of image pairs composed of optical and synthetic aperture radar (SAR) images is a highly challenging task because of the inherently different physical, statistical, and textural properties of the input data. Information-theoretic measures capable of comparing local intensity distributions are often used for multisensor optical-SAR registration. Moreover, the growing availability of such heterogeneous data from current space missions require multisensor registration methods able to run on large-scale datasets with acceptable computation times. In this paper, a novel method is proposed combining information-theoretic area-based registration with a sequential image tiling strategy. Experiments with optical-SAR data collected by a variety of sensors (Sentinel, Landsat, ERS, etc.) suggest both qualitatively and quantitatively the effectiveness of the proposed strategy in achieving accurate registration with low computational cost

Optical-SAR Decision Fusion with Markov Random Fields for High-Resolution Large-Scale Land Cover Mapping

Decision fusion allows making a common decision by combining multiple opinions. In the context of remote sensing classification, such techniques are of great importance in all the cases where data collected by multiple sensors are merged into a final decision. Decision fusion may be used to combine the posterior probabilities associated with the output of single classifiers when applied to single sensor data. Meanwhile, techniques such as Markov Random Fields (MRFs) can integrate contextual information in the fusion process and are commonly used in classification. However, in the context of very large scale mapping (e.g., for global climate change monitoring), computation time can be critical and the application of both data fusion and spatial-contextual modeling comes with several constraints. In this paper, we propose a Bayesian decision fusion approach for optical-SAR image classification, integrated with a fast formulation of the iterated conditional modes (ICM) MRF-optimization algorithm based on a convolution operation. he validation on wide areas of Siberia proved the scalability and efficiency of the method for large scale applications

Multiresolution and Multimodality Sar Data Fusion Based on Markov and Conditional Random Fields for Unsupervised Change Detection

Current satellite missions (e.g., COSMO-SkyMed, Sentinel-1) collect single- or multipolarimetric synthetic aperture radar (SAR) images with multiple spatial resolutions and possibly short revisit times. The availability of heterogeneous data requires effective methods able to exploit all the available information. In the context of environmental monitoring and natural disaster recovery, this paper proposes an unsupervised change detection method able to properly fuse and exploit multiresolution and multimodality SAR data. The data fusion process is based on the estimation of the virtual images that would have been collected in case all the sensors worked at the same spatial resolution and on the definition of a probabilistic model based on generalized Gaussian distributions and Gram-Charlier approximations. The detection of changes is addressed in a probabilistic graphical framework through a novel conditional random field, by defining an energy function that is minimized through graph-cuts or belief propagation methods

Automatic Area-Based Registration of Optical and SAR Images Through Generative Adversarial Networks and a Correlation-Type Metric

The automatic registration of multisensor remote sensing images is a highly challenging task due to the inherently different physical, statistical, and textural properties of the input data. In the present paper, this problem is addressed in the case of optical-SAR images by proposing a novel method based on deep learning and area-based registration concepts. The method integrates a conditional generative adversarial network (cGAN), an area-based cross-correlation-type ell^{2} similarity metric, and the COBYLA constrained maximization algorithm. Whereas correlation-type metrics are typically ineffective in the application to multisensor registration, the proposed approach allows exploiting the image translation capabilities of cGAN architectures to enable the use of an ell^{2} similarity metric, which favors high computational efficiency. Experiments with Sentinel-1 and Sentinel-2 data suggest the effectiveness of this strategy and the capability of the proposed method to achieve accurate registration

EXPERIMENTAL COMPARISON OF REGISTRATION METHODS FOR MULTISENSOR SAR-OPTICAL DATA

Synthetic aperture radar (SAR) and optical satellite image registration is a field that developed in the last decades and gave rise to a great number of approaches. The registration process is composed of several steps: feature definition, feature comparison and optimization of a geometric transformation between the images. Feature definition can be done using simple traditional filtering or more complex deep learning (DL) methods. In this paper, two traditional approaches and a DL approach are compared. One can then wonder if the complexity of DL is worth to address the registration task. The aim of this paper is to quantitatively compare approaches rooted in distinct methodological areas on two common datasets with different resolutions. The comparison suggests that, although more complex, the DL approach is more precise than traditional methods

Hepatic Elastometry and Glissonian Line in the Assessment of Liver Fibrosis

The aim of this study was to identify a method for staging hepatic fibrosis using a non-invasive, rapid and inexpensive technique based on ultrasound morphologic hepatic features. A total of 215 patients with different liver diseases underwent B-mode (2-D brightness mode) ultrasonography, vibration-controlled transient elastography, 2-D shear wave elastography and measurement of the controlled attenuation parameter with transient elastography. B-Mode images of the anterior margin of the left lobe were obtained and processed with automatic Genoa Line Quantification (GLQ) software based on a neural network for staging liver fibrosis. The accuracy of GLQ was 90.6% during model training and 78.9% in 38 different patients with concordant elastometric measures. Receiver operating characteristic curve analysis of GLQ performance using vibration-controlled transient elastography as a reference yielded areas under the curves of 0.851 for F 65 F1, 0.793 for F 65 F2, 0.784 for F 65 F3 and 0.789 for F 65 F4. GLQ has the potential to be a rapid, easy-to-perform and tolerable method in the staging of liver fibrosis